The proliferation of computers and the advent of the Internet and the maturing of the World Wide Web (“web”) have resulted in the development of new and improved software applications for both business and personal uses. Examples of software applications include collaboration applications, chat room applications, instant messaging applications, conferencing applications, gaming applications, and the like. These software applications are often tested prior to commercial release in order to ensure proper operation of the software applications.
One aspect of software testing focuses on ensuring reliable operation of a software application under anticipated loads. Load testing is a process by which a software application is tested under load—i.e., stress—that the software application is likely to experience in a real operation environment. Load tests are end-to-end performance tests that test the application program under an anticipated production load, which is typically based on a prediction of how the end users would use the software application. One objective of the load tests is to determine the response times for various operations performed by the application program. Another objective of the load tests is to measure the capability of the application program to function correctly under load.
There are a number of traditional approaches to load testing. One approach is to employ live users to load test the application program. The application program developer creates a laboratory with computers and communications equipment, and hires live users to access the application program in order to drive the application program under load. A major drawback to this approach is its prohibitive cost. To properly test the application program under anticipated loads may require the use of hundreds or even thousands of computers and live users, who will need to be trained in the operation and use of the application program. In addition, test scripts may need to be prepared for each live user to ensure adequate testing of various functions of the application program. Moreover, test directors may need to be present during the testing in order to coordinate certain aspects of the load testing.
Another approach is to use test scripts to load test the application program. Test scripts are commonly written using a “scripting” language and usually require the use of an interpreter. The interpreter links to the application program, typically a programming interface of the application program, and sends commands to the application program in accordance with the commands in the test script. A test developer or developers who are familiar with the scripting language and the application program prepare the test scripts for load testing the application program. The test scripts are then tested to ensure proper operation, and if errors are found or the test scripts do not operate correctly, the test developer revises or corrects the test scripts. The test scripts are then loaded onto a large number of machines, typically in a laboratory, and executed to load test the application program. A major drawback to this approach is that it is very expensive. To properly develop the test scripts requires a significant amount of the test developers' time, which tends to be very expensive. Another drawback is that the test scripts do not provide an adequate mechanism for interaction between the test scripts, thus making it very difficult to coordinate the testing of the features of the application program. Still another drawback is the static nature of the test commands contained in the test scripts. These test commands are typically hard coded in the test scripts, and in order to vary the commands and/or the load, the test scripts need to be changed.
To address some of the drawbacks of the traditional load testing approaches, performance and load testing tools have been developed. While many of these tools provide the ability to create hundreds or thousands of virtual users to apply consistent and repeatable loads on an application program, these tools are inflexible in that they do not allow automatic variations in the load. For example, these tools require manual intervention, for example, by a test developer familiar with the operation of the load testing tool, to vary the number of virtual users or the actions performed by the virtual users. Also, much like the traditional test scripts, these tools do not support messaging between the virtual users, which makes synchronizing and/or altering the load testing almost impossible.
It would be desirable to have a technique that would allow for generating and monitoring variable load on an application program.